Diminished Control in Crowdsourcing: An Investigation of Crowdworker Multitasking Behavior

Obtaining high-quality data from crowds can be difficult if contributors do not give tasks sufficient attention. Attention checks are often used to mitigate this problem, but, because the roots of inattention are poorly understood, checks often compel attentive contributors to complete unnecessary work. We investigated a potential source of inattentiveness during crowdwork: multitasking. We found that workers switched to other tasks every five minutes, on average. There were indications that increasing switch frequency negatively affected performance. To address this, we tested an intervention that encouraged workers to stay focused on our task after multitasking was detected. We found that our intervention reduced the frequency of task-switching. It also improves on existing attention checks because it does not place additional demands on workers who are already focused. Our approach shows that crowds can help to overcome some of the limitations of laboratory studies by affording access to naturalistic multitasking behavior

Citizen scientists are not just quiz takers: Information about project type influences data disclosure in online psychological surveys

Traditionally, citizen science has centred on giving lay people opportunities to learn about science by participating in it. Lately, psychological citizen science projects have increasingly aimed to attract participants by providing an opportunity for self-learning. Unfortunately, these citizen science projects sometimes resemble quizzes hosted by private companies that have been criticised for suboptimal privacy practices. We were interested in whether similarity to these dubious quizzes could jeopardise the ability of citizen science projects to collect sensitive personal data. We found that people who join an online quiz disclose a lower volume of sensitive data than those who join a citizen science project. Moreover, post-recruitment encouragement to learn about science influenced data disclosure among quiz takers but not among citizen scientists. These findings have implications not only for the effective design of citizen science but also for the broader debate about consent to disclose data in research, academic or otherwise

Home is Where the Lab is: A Comparison of Online and Lab Data From a Time-sensitive Study of Interruption

While experiments have been run online for some time with positive results, there are still outstanding questions about the kinds of tasks that can be successfully deployed to remotely situated online participants. Some tasks, such as menu selection, have worked well but these do not represent the gamut of tasks that interest HCI researchers. In particular, we wondered whether long-lasting, time-sensitive tasks that require continuous concentration could work successfully online, given the confounding effects that might accompany the online deployment of such a task. We ran an archetypal interruption experiment both online and in the lab to investigate whether studies demonstrating such characteristics might be more vulnerable to a loss of control than the short, time-insensitive studies that are representative of the majority of previous online studies. Statistical comparisons showed no significant differences in performance on a number of dimensions. However, there were issues with data quality that stemmed from participants misunderstanding the task. Our findings suggest that long-lasting experiments using time-sensitive performance measures can be run online but that care must be taken when introducing participants to experimental procedures

Assessing the Viability of Online Interruption Studies

Researchers have been collecting data online since the early days of the Internet and as technology improves, increasing numbers of traditional experiments are being run online. However, there are still questions about the kinds of experiments that work online, particularly over experiments with time-sensitive performance measures. We are interested in one time-sensitive measure specifically, the time taken to resume a task following an interruption. We ran participants through an archetypal interruption study online and in the lab. Statistical comparisons showed no significant differences in the time it took to resume following an interruption. However, there were issues with data quality that stemmed from participant confusion about the task. Our findings have implications for experiments that assess time-sensitive performance measures in tasks that require continuous attention

Shortlinks and tiny keyboards: a systematic exploration of design trade-offs in link shortening services

Link-shortening services save space and make the manual entry of URLs less onerous. Short links are often included on printed materials so that people using mobile devices can quickly enter URLs. Although mobile transcription is a common use-case, link-shortening services generate output that is poorly suited to entry on mobile devices: links often contain numbers and capital letters that require time consuming mode switches on touch screen keyboards. With the aid of computational modeling, we identified problems with the output of a link-shortening service, bit.ly. Based on the results of this modeling, we hypothesized that longer links that are optimized for input on mobile keyboards would improve link entry speeds compared to shorter links that required keyboard mode switches. We conducted a human performance study that confirmed this hypothesis. Finally, we applied our method to a selection of different non-word mobile data-entry tasks. This work illustrates the need for service design to fit the constraints of the devices people use to consume services

Watching movies on netflix: Investigating the effect of screen size on viewer immersion

Film and television content is moving out of the living room and onto mobile devices - viewers are now watching when and where it suits them, on devices of differing sizes. This freedom is convenient, but could lead to differing experiences across devices. Larger screens are often believed to be favourable, e.g. to watch films or sporting events. This is partially supported in the literature, which shows that larger screens lead to greater presence and more intense physiological responses. However, a more broadly-defined measure of experience, such as that of immersion from computer games research, has not been studied. In this study, 19 participants watched content on three different screens and reported their immersion level via questionnaire. Results showed that the 4.5-inch phone screen elicited lower immersion scores when compared to the 13-inch laptop and 30-inch monitor, but there was no difference when comparing the two larger screens. This suggests that very small screens lead to reduced immersion, but after a certain size the effect is less pronounced

Using nature-based soundscapes to support task performance and mood

Smartphone apps that enable workers to listen to nature soundscapes are increasingly popular. There is, however, little evidence that these soundscapes have the effects that they claim to have. Previous research exploring the effect of listening to background music during tasks has shown that while such music may have a positive effect on emotional state, it can disrupt reading and memory-based tasks. This paper explores the effects of nature soundscapes on mood and performance. A diary study of the use of soundscapes whilst studying suggests that students view such soundscapes as: aiding focus whilst studying; creating feelings of calm and peace; helping to manage stress and anxiety; and hiding distracting sounds. A second study - an experiment - investigated the effects of nature soundscapes on mood and performance. Whilst we found no effect of soundscapes on mood and arousal during the task, our results demonstrate that high acoustic variation in a soundscape may cause a disruption to serial recall tasks. The implications of our findings suggest that nature soundscapes with high acoustic variation may be detrimental to task performance compared to working in silence for serial based thinking tasks

Understanding people: A course on qualitative and quantitative HCI research methods

This course will provide an introduction to methods used in Human-Computer Interaction (HCI) research. An equal focus will be given to both the quantitative and qualitative research traditions used to understand people and interactional contexts. We shall discuss these major research traditions along with their contemporary framings (e.g., in-the-wild research and Interaction Science). By the end of the course attendees will have a detailed understanding of how to select and apply methods to address a range of problems that are of concern to contemporary HCI researchers

Research Methods for HCI: Understanding People Using Interactive Technologies

This course will provide an introduction to methods used in Human-Computer Interaction (HCI) research. An equal focus will be given to both the quantitative and qualitative research traditions used to understand people and interactional contexts. We shall discuss these major philosophical traditions along with their contemporary framings (e.g., in-the-wild research and Interaction Science). By the end of the course attendees will have a detailed understanding of how to select and apply methods to address a range of problems that are of concern to contemporary HCI researchers